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Specification 

1 . The disclosure is objected to because of the following informalities: 
Page 1 , line 21 the phrase "in one aspect" ought to be deleted. 

Page 5, line 20 the phrase "image 330" aught to be changed to 340 for a proper 
correspondence with the subject matter. 
Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shum et al. (US Patent 6,643,413 B1) and in view of Heckbert (Master's Thesis from 
the IDS). 

Regarding claim 1, Shum et al. discloses a method for shifting perspective (Fig. 
1 1 uses image based rendering as shown in figure 2 that allows a user to "desirably 
able to move and rotate a virtual camera 204 throughout the scene 200, as indicated by 
the arrows extending from the scene 204, so that the virtual camera represents a 
desired viewpoint of the object 202. An image representing this viewpoint of the object 
202 is then rendered (col. 5, lines 55-60).") in a composite image derived from a 
plurality of images (Figure 6 is a mosaic image of 8 images with eight rays that 
represent depth information of images as mentioned in col. 1, lines 47-49. Note that 
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rays and images are used to represent an image with depth information because the 
rays or depth information when available provide additional information for the images 
as mentioned in col. 1 , lines 37,38.) including a first image (The ray of figure 6 with the 
angle 6 represents depth information of a first image on the mosaic 602.) and a second 
image (The ray of figure 6 pointing vertically downward is a second image.), the 
composite image including the first image as a center of projection (A virtual camera 
located in the center of figure 6 is positioned, in the step of figure 1 1 ,num. 1 104, on the 
first image as the current position as mentioned in col. 13, lines 34,35 for rendering and 
displaying an image in the steps of fig. 11, numerals 1106 and 1108, respectively.) and 
a modified version of the second image (The second image is warped or modified to the 
first image, in the step of fig. 1 1 , num. 1 106, because the second image's 
corresponding ray is closer to the first image's corresponding ray as mentioned in col. 
1 , lines 37-42.), the modified version of the second image being corrected for 
perspective distortion relative to the first image (The vertically downward ray of figure 6 
represents depth information or perspective information that is warped to the first image 
for a continuous motion between views mentioned in col. 9, lines 46-49.), the method 
comprising: 

a) receiving an instruction to shift perspective (In the step of fig. 1 1 , num. 
1 104, a virtual camera is used to shift from a current position or first image(or the arrow 
with 6 in fig. 6.) on a mosaic to a new position or the second image (or the vertically 
downward arrow of fig. 6.) on the same mosaic as mentioned in col. 13, lines 34-35 .) to 
make the second image the center of projection of the composite image (The virtual 
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camera is now positioned in front of the second image. The virtual camera was initially 
positioned in front of the first ray with 6.).; 

b) determining a transformation (or warping is the only transformation used 
as mentioned in col. 1 , lines 40-42 and col. 9, line 46-51 .) for a set of reference points in 
the modified version of the second image (Transforming the mosaic 602 of figure 6 from 
one coordinate space to another as shown in figure 10, the set of reference points in the 
modified version of the second image are shown by the curved line of sampled points 
1006. Note that the horizontal double headed arrow on "r sub. 1" correspond to a 
continuous movement on the mosaic 1006 or the mosaic of figure 6, num. 602, and 
continuous movement corresponds to a warping between other close rays (not shown in 
figure 10 and shown in figure 6.) as mentioned in col. 9, lines 49-51.) to a corresponding 
set of reference points in second image (The intersection at the point on the composite 
image or curved Iine1006 and ray 1002 is a view of the second image.); and 

c) transforming the first image and the modified version of the second image 
based on the transformation to generate a modified version of the first image and the 
second image. (Once the virtual camera is positioned by a user from the fist image 
(Figure 6 has a ray with e as the first image) to the second image (Figure 6 has a 
vertically downward arrow as the second image) the first image is warped to the second 
image because the first ray is the closest or local ray to the second ray as mentioned in 
col. 1, lines 40-42, col. 8, lines 8,9, col. 9, lines 49-51 and col. 13, lines 41-44.) 
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Shum et al. does not teach a mapping between images, but Shum et al. does 
teach extracting a set of points, such as a first portion of the curved line of fig. 10, num. 
1006) for a corresponding view (or a new view from a second portion of fig. 10, num. 
1006 that is different from the first extraction of points.) as mentioned in col. 8, lines 10- 
12. 

However, Heckburt, in the field of endeavor of image mapping, does teach a 
mapping ("bilinear mapping" of figure 2.5 on page 15) of points (Figure 2.5 on the right 
side has four points labeled u p00,p01,p10 and p11) between images (Figure 2.5 has a 
source image on the left side and a destination image on the right side.) or planes as 
shown in page 15, figure 2.5. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the curve line that represents a mosaic (Fig. 6, 1006) and a ray 
(Fig. 6, num. 1002) that represents the depth of the image of Shum et al. with the 
teaching of Heckburt' s mapping, because Heckburt's mapping provides a predictable 
transformation between images. Therefore users in 3D modeling and image distortion 
software can use the mappings to provide predictable features (lines, equispaced 
points, and angles) that are persevered (Heckburt, page 11, Section 2.2). 

Regarding claim 2, Shum et al. and Heckburt teaches the method of claim 1 , 
further comprising: 

merging the modified version of the first image and the second image to form a 
second composite image that has the second image as its center of projection. The 
step of figure 1 1 , num. 1 1 08 renders the new image using the second image with the 
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first image modified or warped to the second image upon the movement of the virtual 
camera that creates a current positioned that is positioned in front of the second image 
that has a the first image warped to the second image. 

Regarding claim 3, Shum et al. and Heckburt teaches a third image (Fig. 6 has a 
ray in the diagonal bottom right position that represents the third image.) that is warped 
in the same way as first image of claim 1 because the third image is also a ray that is 
closest to the second image (the vertically downward ray.) when the virtual camera is * 
positioned in front of the second image. 

Regarding claim 4, Shum et al. and Heckburt teaches the method of claim 1 , . 
wherein: 

a) the reference points in the modified version of the second image include 
four non-collinear and non-coincident points in the modified version of the second image 
(Heckburt, figure 2.8 is a projective mapping of four corner points (labeled 0,1,2 and 3 of 
the quadrilateral or trapezoid figure on the left side.) that are in different positions and 
form a non-linear line that is modified from the corresponding middle figure.; and 

b) the reference points in the second image include four non-collinear and 
non-coincident points in the second image as shown in Heckburt, fig. 2.8, middle figure 
is a square or rectangle. 

Regarding claim 5, Shum et al. and Heckburt teaches the method of claim 1 , 
wherein: 

a) the second image and the modified version of the second image each 
include a perimeter as shown by the bold lines of figure 2.8 of Heckburt. 
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b) the reference points in the second image and the modified version of the 
second image are vertices on the perimeters on the second image and the modified 
version of the second image. Heckburt states," In an interactive image warper one might 
specify the four corners of source and destination quadrilaterals... and wish to warp one 
area to the other... [as mentioned in claim 4] (Heckburt, page 19, first sentence of the 
section "Inferring Projective Mappings". Figure 2.8 of Heckburt shows how the four 
vertices are transformed from case 1 to case 3.) 

Regarding claim 6, Shum et al. and Heckburt teaches the method of claim 5, 
wherein: 

a) the first image includes a plurality of pixels and has a perimeter that 
defines a set of vertices which was addressed in claim 5; and 

b) transforming the first image based on the transformation includes: 
b1 ) transforming the vertices of the first image (Using fig 2.8 of 

Heckburt, case three is shown where the vertices of the figure on the left side of figure 
2.8 is transformed to the right-hand figure of figure 2.8.); and 

b2) transforming the pixels of the first image based on the 
transformation of the vertices which was addressed in claim 5. 

Regarding claim 7, Shum et al. and Heckburt teaches the method of claim 6, 
wherein: 

The transformation is represented as a transformation matrix or (M sub. sd) is a 
forward mapping matrix as mentioned in Heckburt, page 19, section:" Inferring 
Projective Mappings", line 7. 



Application/Control Number: 09/941 ,427 Page 8 

Art Unit: 2621 

Regarding claim 8, Shum et al. and Heckburt teaches the method of claim 7, 
wherein: 

The transformation matrix is derived from the vertices of the modified version of 
the second image. Heckburt teaches that all points in one image (u sub. k, v sub. k) is 
mapped to other points (x sub. k, y sub. k) in a corresponding image using vertices 
(k=0,1,2 t 3)for each point. 

Claim 9 was addressed in claim 8. 

Regarding claim 10, Shum et al. and Heckburt teaches the method of claim 9, 
wherein the transformation matrix,M, is given by the matrix equation shown in Heckburt, 
page 19. 

Claims 1 1 and 13 have been addressed in claim 4. 

Regarding claim 12 a reference point ("1") in the left-hand figure shown in 
Heckburt, fig. 2.8 is moved to another position as shown in the figure of 2.8 on the 
right-hand side is move relative to other reference points 0,2, and 3 to change the 
shape of the figure on the left side of figure 2.8 to the figure on the right-hand side of 
figure 2.8. 

Regarding claim 14, Shum et al. and Heckburt teaches the method of calim 1, 
wherein: 

a) the instruction to shift perspective is received as a single user input(A user 
moves a virtual camera(Shum, fig. 2, num. 204) during the step of figure 11, num. 1104 
of Shum et al. to change the current view to a new view as mentioned in Shum et al. 
col. 13, lines 34,35; and 
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b) the determining and transforming steps are automatically performed in 
response to the user input (A user is able to move and rotate the virtual camera that has 
a desired view of an object. The desired view is "then rendered" as mentioned in Shum 
et al., col. 5, lines 55-60. Thus as the virtual camera's current view is moved a 
corresponding image view is rendered using the method of figure 11.) 

Regarding claim 15, the combination of Shum et al. and Heckburt teaches all the 
limitations as discussed in claims 1, 2, 3, and 14 and a computer-implemented image 
processing method (Shum et al., fig. 1 is a computer that processes the method of fig. 
11 of Shum etal.). 

Regarding claim 16, the combination of Shum et al. and Heckburt teaches all the 
limitations as discussed in claims 1 , 2 and 3 and a computer program product, tangibly 
stored on a computer-readable medium (Shum et al., fig. 1 , num. 29,31 ) that is provided 
to a processor (Shum et al., fig. 1,num. 21). 

Claim 17 has been addressed in claim 2. 

Claim 18 has been addressed in claim 3. 

Claim 19 has been addressed in claim 4. 

Claim 20 has been addressed in claim 5. 

Claim 21 has been addressed in claim 6. 

Claim 22 has been addressed in claim 7. 

Claim 23 has been addressed in claim 8. 

Claim 24 has been addressed in claim 9. 

Claim 25 has been addressed in claim 10. 
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Claim 26 has been addressed in claim 1 1 . 
Claim 27 has been addressed in claim 12. 
Claim 28 has been addressed in claim 13. 
Claim 29 has been addressed in claim 14. 

Regarding claim 30, the combination of Shum et al. and Heckburt teaches all the 
limitations as discussed in claims 1, 2, 3, 14 and 16. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Yano et al (US Patent 6,714,689 B1) is pertinent as teaching a method of 
aligning multiple images to a reference image as shown in figure 3. 

Peleg et al. (US Patent 6,532,036 B1) is pertinent as teaching a method of 
creating a mosaic(fig. 6, num. 604) with one perspective point from multiple camera 
views (fig. 6, num. 601-603). 

Shum et al. (US Patent 6,084,592 A) is pertinent as teaching a method of viewing 
a room using a 3D model (fig. 10N) and a mosaic image (fig. 10A) inlaid on the interior 
of the 3D model of figure 1 0B. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario-Vasquez whose telephone number is 
703-305-5431 . The examiner can normally be reached on 9-5. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on 703-305-4706. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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Unit 2621 
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